Move pod jobs to parallel execution #7382
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: mheon The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci-robot
added
the
approved
mheon
force-pushed
the
pod_parallel
branch
4 times, most recently
from
93b045a to
51d10eb
Compare
mheon
force-pushed
the
pod_parallel
branch
2 times, most recently
from
51b555a to
2684f0f
Compare
jwhonce
reviewed
Sep 17, 2020
pkg/parallel/parallelctr/ctr.go
Outdated
| @@ -0,0 +1,42 @@ | |||
| package parallelctr | |||
There was a problem hiding this comment.
Why stutter name on enclosing directory and file?
There was a problem hiding this comment.
Because I am not very creative and could not think of a better name.
mheon
force-pushed
the
pod_parallel
branch
4 times, most recently
from
230a495 to
e2c8595
Compare
Make Podman pod operations that do not involve starting containers (which needs to be done in a specific order) use the same parallel operation code we use to make `podman stop` on large numbers of containers fast. We were previously stopping containers in a pod serially, which could take up to the timeout (default 15 seconds) for each container - stopping 100 containers that do not respond to SIGTERM would take 25 minutes. To do this, refactor the parallel operation code a bit to remove its dependency on libpod (damn circular import restrictions...) and use parallel functions that just re-use the standard container API operations - maximizes code reuse (previously each pod handler had a separate implementation of the container function it performed). This is a bit of a palate cleanser after fighting CI for two days - nice to be able to return to a land of sanity. Signed-off-by: Matthew Heon <[email protected]>
Ensure that we actually print the output of all commands when cleaning up the results of the E2E tests. Signed-off-by: Matthew Heon <[email protected]>
|
This should (finally) go green now |
|
It's green. @baude @rhatdan @vrothberg @TomSweeneyRedHat @jwhonce PTAL |
| }() | ||
|
|
||
| return retChan | ||
| } |
There was a problem hiding this comment.
Really neat function. At first I was looking all over above for the channel close.
|
LGTM |
|
/lgtm |
github-actions
bot
added
the
locked - please file new issue/PR
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Make Podman pod operations that do not involve starting containers (which needs to be done in a specific order) use the same parallel operation code we use to make
podman stopon large numbers of containers fast. We were previously stopping containers in a pod serially, which could take up to the timeout (default 15 seconds) for each container - stopping 100 containers that do not respond to SIGTERM would take 25 minutes.To do this, refactor the parallel operation code a bit to remove its dependency on libpod (damn circular import restrictions...) and use parallel functions that just re-use the standard container API operations - maximizes code reuse (previously each pod handler had a separate implementation of the container function it performed).
This is a bit of a palate cleanser after fighting CI for two days - nice to be able to return to a land of sanity.